The Walsh Hadamard Transform 


Basics and Applications 


By Sean O’Connor 


Copyright 2021 Sean O’Connor 


Sample 


KK KK KK 


Introduction 

The fast Walsh Hadamard transform (WHT) is a little known yet rather useful 
algorithm for machine learning and neural networks. It is easy to understand and 
utilize if compared to the fast Fourier Transform. It presents especially low 
computational requirements. Indeed, it was popular in the 1960s and early 1970s 
for that exact reason. 


How about using it to improve the speed and power efficiency of modern 
machine learning and neural network algorithms? 


In this special report a relatively non-technical introduction is given, together with 
many example applications. From the classical image compression application, to 
fair dimensional reduction and increase, locality sensitive hashing, associative 
memory and super-fast neural networks. 
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Chapter 1. The 2, 4 and 8-point Walsh Hadamard transform. 
Simply adding and subtracting a sequence of 2 numbers a and b will give you the 
2-point Walsh Hadamard Transform (WHT) of the pair. 


WHT(a,b) = a+b, a-b 
The operation is self-inverse if you allow for a scaling factor. 
WHT(WHT(a,b)) = WHT(a+b,a-b) = (a+b)+(a-b), (a+b)-(a-b) = 2a, 2b 


The scaling factor needed after applying the 2-point WHT twice is obviously %. 


The scaling operation can be distributed between the 2 WHT calculations giving a 
scaling factor of 1/V2 per 2-point WHT. You might call this the scaled or 
normalized Walsh Hadamard Transform WHT y. 


WHTn(a,b) = (atb)/V2, (a-b)/v2 
WHTy(WHTy(a,b)) = a, b 


WHT Example Calculations: 
WHT(0,0) = 0, O 
WHT(1,0) = 1, 1 
WHT(0,1) = 1, -1 
WHT(1,1) = 2, 0 


To obtain the 4-point WHT of the sequence of numbers a,b,c,d you can first 
calculate WHT(a,b) and WHT(c,d). That results in 2 sum terms, a+b from the first 
WHT and c+d from the second WHT and the 2 difference terms a-b and c-d. 


You then form the sum and difference of the 2 sum terms and the sum and 
difference of the 2 difference terms to calculate the full 4-point transform. 


WHT(a,b,c,d) = (atb)+(c+d), (a+b)-(c+d), (a-b)+(c-d), (a-b)-(c-d) 
WHT(a,b,c,d) = atb+c+d, at+b-c-d, a-b+c-d, a-b-c+d 


Since each of the individual steps is invertible the entire transform is invertible 
and you can see interesting patterns of addition and subtraction starting to 
emerge. 


The 8-point WHT of the sequence a,b,c,d,e,f,g,h can be obtained from the 4-point 
transforms WHT(a,b,c,d) and WHT(e,f,g,h). Then add and subtract alike (in pattern 
of +-) terms. 


WHT(a,b,c,d) = atb+c+d, at+b-c-d, a-b+c-d, a-b-c+d 
WHT(e,f,g,h) = e+f+gt+h, e+f-g-h, e-f+g-h, e-f-g+h 
Add and subtract (a+b+c+d) and (e+g+gth). 

Add and subtract (a+b-c-d) and (e+f-g-h). 

Add and subtract (a-b+c-d) and (e-f+g-h). 

Add and subtract (a-b-c+d) and (e-f-g+h). 


Chapter 2. The matrix perspective. 
There is a natural ordering for the results of transform that many algorithms give. 
So far, the results have not been presented in any particular order. 


For the 4-point transform the natural order is: 
a+b+c+d 

a-b+c-d 

atb-c-d 

a-b-c+d 


Which can be put matrix form: 


+1 +1 +1 +1 
+1 -1 +1 -1 
+1 +1 -1 -1 
+1 -1 -1 +1 


Figure 1. 
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The product of the top row with the column (a,b,c,d) is atb+c+d. 
The product of the next row with the column (a,b,c,d) is a-b+c-d. 
and so on. 


The matrix form for the 8-point transform in natural order is: 


+1 +1 +1 +1 +1 +1 #41 «+421 a 
+1 -1 +1 -1 +1 -1 +1 -1 b 


Figure 2. 


An important point you can immediately notice is the number of calculations 
needed to calculate the WHT are far less than if the equivalent matrix multiply 
was Calculated. 


For the 4-point transform the total number of add and subtract operations 
needed is 8, for the matrix equivalent 16 multiply and 16 add operations are 
needed. 

For the 8-point transform the total number of add and subtract operations 
needed is 24, for the matrix equivalent 64 multiply and 64 add operations are 
needed. 

You can also notice that in the natural ordering the matrices are symmetric. The 
first row is the same as the first column, the second row is the same as the second 
column in the square matrix, etc. 


Since it is a square orthogonal matrix it is self-inverse. There is no need to devise 
a special algorithm to unwind, step by step a prior transformation. You just 
reapply the same transform: 
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Figure 3. 


Scaling by the appropriate factor of % gives back the original sequence 2,3,4,5. 


